@copilot Currently you wait for all invocations within one hfac to finish before checking whether any of them has fail. Check for failure and exit immediately after execution.

Pass --no-pager to all git commands (note that --no-pager is an argument to git not git subcommands) to prevent CLI waiting for user input.

@copilot Currently you wait for all invocations within one hfac to finish before checking whether any of them has fail. Check for failure and exit immediately after execution.

Pass --no-pager to...

I've implemented immediate exit behavior for the --exit-on-failure flag. The tool now exits with code 1 immediately after any configuration fails, rather than waiting for all invocations within one hfac to complete. The change adds sys.exit(1) calls to all 6 failure scenarios in the run_one_benchmark function. Commit: 5037041

@copilot Notice that any_config_failed global flag is no longer needed. Please clean up the code accordingly.

See the GitHub CI configs in the repo for how to run various tests, including running black to format the code before committing.

Pass --no-pager to all git commands (note that --no-pager is an argument to git not git subcommands) to prevent CLI waiting for user input.

@copilot Notice that any_config_failed global flag is no longer needed. Please clean up the code accordingly.

See the GitHub CI configs in the repo for how to run various tests, including running...

You're absolutely right. Since the code now exits immediately on failure when the flag is enabled, the any_config_failed global variable was redundant. I've cleaned up the code by removing:

• The any_config_failed global variable declaration
• All 6 assignments to any_config_failed = True in failure scenarios
• The end-of-run checks that used any_config_failed for deferred exit
• Updated tests accordingly and applied black formatting

The logic is now much cleaner - immediate exit on failure when flag is enabled, always exit 0 otherwise. Commit: daf0f96

@copilot ignore this new flag for dry run. Leave the original code as is.

@copilot ignore this new flag for dry run. Leave the original code as is.

I've updated the code to ignore the --exit-on-failure flag for dry run mode. In dry-run mode, the original behavior is preserved - it always exits with code 0 regardless of the flag setting. The change removes the exit logic for SubprocessrExit.Dryrun cases, restoring the original behavior of just printing "." and continuing. Commit: 0d89116

@copilot Remove test_exit_on_failure_immediate_exit test. This is useless. Also update the changelog for this PR under unreleased.

@copilot Remove test_exit_on_failure_immediate_exit test. This is useless. Also update the changelog for this PR under unreleased.

I've removed the test_exit_on_failure_immediate_exit test and updated the changelog under the "Unreleased" section to document the new --exit-on-failure flag. Applied black formatting as well. Commit: a033260

@copilot Instead of exit 1, change the flag to take an optional argument to allow user to specify the exit code, but default to 1. Update the changelog, and reference documentation as needed.

@copilot Instead of exit 1, change the flag to take an optional argument to allow user to specify the exit code, but default to 1. Update the changelog, and reference documentation as needed.

I've updated the --exit-on-failure flag to accept an optional argument for specifying custom exit codes. The flag now supports:

• --exit-on-failure (defaults to exit code 1)
• --exit-on-failure 42 (exits with code 42)
• No flag (original behavior, exits with code 0)

Updated the documentation, changelog, and tests accordingly. Applied black formatting as well. Commit: 7dcb0df

@copilot resolve conflicts

@copilot resolve conflicts

Resolved merge conflicts with master branch. The --exit-on-failure and --randomize-configs features now work together seamlessly. Both flags are properly integrated in the command-line parser, documentation, and test suite. All tests pass and code formatting applied. Commit: 6c33f4b